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CHAPTER 2 

TERRESTRIAL APPLICATIONS: AN INTELLIGENT EARTH-SENSING 

INFORMATION SYSTEM 


opment of an integrated, user-oriented, Earth-sensing infor- 
mation system (fig. 2.1) incorporating a maximum of 
artificial intelligence capability for two primary reasons. 

First, substantial economic benefits may accrue from the 
effective use of an integrated, intelligent remote Earth- 
sensing system. For example, a reduction in weather 
damage to crops, the location of mineral deposits and earth- 
quake faults, and more efficient means of surveying large 
tracts of land may save time, money, and even human lives. 
With superior definition of models for weather forecasting, 
climate and oceanic processes may eventually make possible 
more precise meteorological prediction and ultimately even 
weather control and global climate modification {Outlook 
for Space, 1976). Such an intelligent sensing system can 
play a dominant role in the activity of understanding the 
Earth as a dynamic physical entity, and can provide a major 



Figure 2.1.- Terrestrial applications: An intelligent Earth-sensing information system. 
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2.1 Introduction 

The Terrestrial Applications Team was charged with 
identifying a sample near-Earth NASA mission that could 
be implemented during the next two or three decades and 
that would require intensive application of artificial intelli- 
gence and robotics technologies. The team initially consid- 
ered a long list of missions that included the design and 
automated fabrication of a satellite solar power station, 
weather sensing and prediction, crop assessment, large 
communication satellites, and disaster monitoring. As the 
catalogue of possible tasks evolved, it became clear that 
artificial intelligence would be most useful when applied to 
missions that generate data at very high rates — such as the 
NASA applications (Landsat) satellites which provide 
imaging data of the Earth. The team focused on the devel- 




part of the ongoing monitoring of Earth useful in effective 
management of the individual and collective activities of 
man. 

Second, NASA currently is obtaining and storing data 
from Earth-sensing satellites at a rate far out of proportion 
to the present or expected utilization of that data. The 
potential utility of collected data is not being realized 
because the raw data are not accessible in a timely and con- 
venient manner, and because most potential users do not 
have the resources to extract useful information from the 
raw files. The current philosophy of data collection and 
storage had its origin in the early days of space research 
when sensors were sent into space, turned on, and all 
results transmitted back and stored. While this appears to 
maximize the utilization of the space vehicle, it has proven 
to be a false economy - the vast majority of uncategorized, 
generally unorganized data have never been and possibly 
never will be analyzed. The data format, its raw condition 
(digital conversions of analog sensor readings), and the 
complete lack of cross-referencing of contents make the 
data extremely difficult to find, interpret, or use. The tre- 
mendous volume of information already amassed and the 
expected increases in future rates of collection due to 
improved sensor technology make the philosophy of 
unorganized data acquisition obsolete. An alternate philos- 
ophy of goal-oriented data collection (information is 
gathered to meet specific objectives) was taken as the 
cornerstone of the proposed mission. 

Thus, the main mission objective was to develop the 
concept of a flexible, intelligent, user-oriented automated 
information system for the collection, analysis, storage, and 
delivery of satellite Earth-sensing information (table 2.1). 


TABLE 2.1.- RATIONALE FOR DEVELOPMENT 
OF AN INTELLIGENT EARTH-SENSING 
INFORMATION SYSTEM. 


Why use remote sensing of the Earth 
Management — control 
Improved understanding — knowledge 
Information cannot be obtained any other way 

Current difficulties 

Vast amount of unorganized data 

Acquisition and distribution of useful information 

High cost 

The solution 

Goal-oriented observation 
Direct user interaction with the system 
World model-based observations 
Autonomous system 


Within rational cost bounds, the system should maximize 
the utilization of this information for the following pur- 
poses: scientific, managerial, commercial, and humani- 
tarian. In addition, the collection and storage of data having 
little or no utility should be minimized, and the costs of 
acquiring, interpreting, and storing Earth resource informa- 
tion must also be reduced. 

Inexpensive data delivery can be accomplished by a sys- 
tem operating with relatively little human intervention. 
Price reduction requires that images be processed without 
costly manual procedures, and that the physical satellite 
system be managed so as to obtain a maximum of useful 
data for the given configuration of orbits and sensors. It 
seems possible to design and construct, by the year 2000, 
a largely autonomous system that can directly interface 
with individual users in natural language, accept requests 
for information, and provide answers based on satellite 
observations coupled with a resident theoretical model of 
the state of the world. Such a system should be able to 
achieve sophisticated data interpretation at modest cost 
through advances in machine hardware and artificial intelli- 
gence techniques. Table 2.2 lists several desirable system 
characteristics and suggested methods for their achieve- 
ment. The key to the proposed system is a sophisticated 
world model (section 2.3) that enables the system to per- 
ceive both the present state of the world and how that state 
changes in time. 


TABLE 2.2.- DESIRABLE CHARACTERISTICS 
OF AN INTELLIGENT EARTH-SENSING 
INFORMATION SYSTEM AND METHODS OF 
REALIZATION. 


Desirable system 

Methods of 

characteristics 

realization 

Cost minimization rela- 

Maximize system 

tive to level of service 

autonomy 

provided 

Interface users directly 
to system 


Goal-oriented observing 
relative to world model 

Wide utilization 

Flexible user interfac- 
ing including natural 
language 

Automatic data 

AI techniques based on 

interpretation 

world model 
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2,1.1 Relationship to NEEDS 

NASA has instituted the NASA End-to-End Data System 
(NEEDS) program, the goal of which is to improve the 
effectiveness and efficiency of the agency’s data and 
information management methodology, NEEDS began with 
Phase I which addressed some very near-term data handling 
and processing problems. Phase II, initiated in 1978, 
concentrated on complete subsystems development to 
permit nearly real-time data management. Future Phase III 
will concentrate on low-cost communications and data 
distribution, and Phase IV will deal with integration of 
modular subsystems and systems techniques. NEEDS is a 
complex program which evolved on a problem-by-problem 
basis to accommodate everincreasing demands placed on it 
by the changing nature of the space program. A summary 
of Phases I and II projects appears in table 2.3. 

TABLE 2.3.- NASA END-TO-END DATA SYS- 
TEM (NEEDS) PROGRAM. 


Phase I 

1 . Synthetic aperture radar processor 

2. Multispectral data processor 

3. Digital data systems 

4. Multipurpose user oriented software techniques 

5. Resource effective data system definitions 

Phase II 

1 . Systems analysis and integration 

2. Modular data transport systems 

3. Information adaptive systems 

4. Database management systems 

5. Archival mass memory 

6. Massively parallel processor 


The intelligent Earth-sensing information system pro- 
posed in this report, and also the Titan mission described in 
chapter 3, will also place significantly increased demands on 
the present NEEDS program. This is not surprising as both 
may be viewed as parts of a natural evolution of present or 
near-term planned NASA missions. Moreover, with these 
new proposals as goals, the NEEDS program can implement 
Phases III and IV in a comprehensive fashion rather than on 
a problem-by-problem basis. The Earth-sensing mission 
demands are at the far extremes of present NEEDS activi- 
ties, particularly in planning, scheduling, and control of 
satellite systems. 


2.2 System Description 

The system and mission goals described in this chapter 
are best summarized as an attempt to propose a flexible, 
ongoing tool of tremendous utility and sophistication. Most 
of the details presented are offered solely to illustrate one 
of many possible alternative approaches. The intent was not 
to prepare an encyclopedic discussion of design specifics 
but rather to indicate the general nature of probable solu- 
tions and provide sufficient subsystem details to permit 
preliminary technology assessment of the basic concept. 

The Intelligent Earth-Sensing Information System 
(lESIS) has the following major features: 

• An intelligent satellite system that gathers data in a 
goal-directed manner, based on specific requests for 
observation and on prior knowledge contained in a 
detailed self-correcting world model (section 2.3). 
The world model eliminates the processing and 
storage of redundant information. 

• A user-oriented interface that permits natural 
language requests to be satisfied without human inter- 
vention from information retrieved from the system 
library or from observations made by a member satel- 
lite within the system. 

• A medium-level onboard decisionmaking capability 
which optimizes sensor utilization without compro- 
mising users’ requests. 

• A library of stored information that provides a com- 
plete detailed set of all significant Earth features and 
resources, adjustable for seasonal and other identi- 
fiable variations. These features and their character- 
istics are accessible through a comprehensive cross- 
referencing scheme. 

lESIS has five major components: (1) System/user 
interface, (2) uplink, (3) satellite sensing and processing, 
(4) downlink, and (5) on-ground processing. The basic 
system is illustrated in figure 2.2. The user connects to the 
on-ground processor via a communication link and inter- 
actively defines his needs with the assistance of the system, 
accessing the database or directing lESIS to collect, process, 
and deliver information as required. The link might be a 
standard telephone line, and the entire transaction could 
occur in keyed natural language. Often, the user request 
should be answerable entirely from intomiation already 
available in the system database, in which case lESIS 
appears to function much like any other interactive 
question-answering system (fig. 2.3). 

Frequently, however, requests will require satellite 
observation data not yet available, in which case appro- 
priate observations are scheduled by the on-ground pro- 
cessor, These instructions are uplinked via geosynchronous 


13 






Figure 2.3. - lESIS response to user request for information already available in the system database. 




communication satellites to orbital lESIS components that 
acquire and process the desired data and return them to the 
ground processor through a downlink (section 2.4,2). The 
ground processing unit further refines the information 
obtained, if necessary, then delivers it to the requester 
(fig. 2.4). 

lESIS nominally operates with only one human in the 
loop - the user. This high degree of autonomy enables the 
system to be cost-effective and capable of rapid response. 
The sequence of operations during any user/IESIS interac- 
tion is outlined in figure 2.5. 

lESIS has two basic modes of operation called back- 
ground and foreground. The background mode of operation 
performs continuous goal-oriented observations of Earth 
and abstracts from these useful information for storage in a 
readily accessible, cross-referenced database. Background 
mode builds a broad scientific knowledge base that provides 
useful historical data at low cost for theory verification and 
testing. The foreground mode allows individual users to 
request that observations be taken and processed in non- 
standard ways. The system must be sufficiently intelligent 
to help “naive users” obtain the information they want in 
an optimal or near-optimal fashion without restricting or 
unduly burdening the more sophisticated user. 



2.2 J Background Mode 

In background mode lESIS continuously observes the 
Earth and gathers information to update the world model 
and to identify anomalies (sensor readings differing signifi- 
cantly from the expected). The system uses its world model 
to eliminate transmission of duplicate data and to imple- 
ment basic principles of management by exception. The 
lESIS world model describes the topography and environ- 
ment of Earth and can predict what a member satellite 
should record during its next observation period. During 
that period, the system collects data for “features” (e.g., 
lakes, forests, coastlines) and identifies all anomalies. 
Feature information is summarized to specify feature status 
without describing every pixel observed. For instance, if the 
height of a lake is known at its inlet and outlet, then the 
lake height at all points and flow rate can be determined. 
Only two pieces of data need be stored and transmitted by 
lESIS, rather than complete data for each pixel of the lake. 

Anomalies are of two types. The first consists of tran- 
sient normal events occurring at random, which are not 
to be permanently included in the world model. Examples 
are ships on an ocean, cars on a road, an iceberg, or a forest 
fire. lESIS should be capable of identifying such events by 


SENSORY DATA 



Figure 2.4. - lESIS response to user request for information that requires satellite observation. 
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Figure 2.5. - Typical data processing steps in any user/ IE SIS interaction. 


their signatures. Observations of these anomalies may pro- 
duce a sample count of the observation type, trigger an 
alarm, or generate a report of the incident automatically 
sent to persons who should be apprised of the situation. 
The second class of anomaly consists of unexpected, wholly 
novel events. Upon observation of such an anomaly (e.g., 
the eruption of Mount Saint Helens’ volcano), all sensor 
data are returned to Earth for analysis, identification, and 
possibly, action. The expected anomaly file is updated to 
include the identity of the phenomenon together with 
directions on actions to take upon re-observation, if any. 

Processed sensor readings for features encountered dur- 
ing an observation are archived. Archival data collected on 
the basis of features and their properties then may be used 
to improve world model accuracy or to build detailed 
models of particular features (e.g., Lake Erie) or types of 
features (e.g., fresh water lakes). Individually observed 
data points lose informational value over time and can be 
reduced to models such as a Fourier time series to retain 
more valuable long-term trends once sufficiently detailed 
surveys accumulate. While the importance of this aspect of 
data reduction will grow over time, the majority of data 
reduction is associated with the world model in the process 


of eliminating the storage of redundant data. The world 
model will enable individual features as well as groups of 
features to be studied and summarized easily. 

2.2.2 Foreground Mode 

The lESIS foreground mode allows individual users to 
make natural language requests for particular data to be 
collected and processed for their own purposes. The ful- 
fillment of this request becomes a goal of the system. lESIS 
determines the appropriate sensor algorithms and requested 
data are acquired the next time the requisite sensors are 
within view of the feature or area to be observed. The 
system must ascertain that conditions specified by the 
requester are satisfied during observation (e.g., absence of 
cloud cover, proper sun angle). If they are not, lESIS 
informs the user and reschedules the run. Nonstandard data 
processing may be performed on sensor data with output in 
any user-specified format including terminal printout, 
photograph/hard copy, and so forth. lESIS must have 
default processing/output modes as well as a choice of 
several optimal preprogrammed methodologies. An unsup- 
ported user-written software library similar to that main- 
tained by IBM also could be provided. 
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2.2,3 User I System Interface 


termination, system files and customer billing records are 
updated. 

The user/system interface illustrated in figure 2.6 has If a user desires satellite scheduling, the optimum 

two basic functions. The first is to process data previously method for obtaining required information including deci- 

coUected and stored on the database to provide desired sions about appropriate sets of sensors, observation fre- 

output. The second is to schedule satellites to make new quencies, sites, and so on, is developed interactively. Cus- 

observations, process data obtained by system routines or tomers are provided initial estimates as to the probable 

user-written procedures, and to notify users when informa- time of completion and expected costs for data collection, 

tion is available for delivery in an appropriate format. and may revise or cancel a request on the basis of these 

Individuals communicate with lESIS through a generally preliminary appraisals. Each such interaction updates the 

accessible information net. After valid identity is estab- request schedule so that observing satellites can perform 

lished, the user indicates via a natural language interface appropriate observations and deliver derived data to the 

whether data retrieval or satellite scheduling is desired. If correct user after processing. Individuals may access the 

this choice is not already known, the system maybe inter- system to cancel or modify a previous request at any time 

rogated regarding relative costs in time or money of using up to the actual taking of data. Further, lESIS can accept 

the most recent file data or the next expected observations. requests for time- or event-oriented measurements such as 

Users requiring data retrieval may interact with lESIS to photography of a particular area every month, or for obser- 

determine the type of information needed and to develop a vations contingent on specific events or conditions 

carefully tailored retrieval and processing scheme. The user recognizable by the system such as volcanic eruptions or 

then ends the session or enters additional requests. Upon forest fires. 



Figure 2.6.- User I system interface. 
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2,2.4 Natural Language Interface 

The primary reason for a natural language interface is to 
allow the largest possible number of novice users to access 
lESIS directly without need for “interpreters.” The princi- 
pal advantage for more knowledgeable customers is con- 
venience, though many customers will likely require a more 
formal and precise interface language in addition to the 
natural language capability. 

There are major problems with current natural language 
interfaces that require careful consideration. The two 
primary difficulties are: 

(1) The machine has only a very rudimentary notion of 
the subject of conversation during sentence interpretation 
even if it is quite competent to answer questions posed in a 
formal query language. 

(2) The machine is not a social being and thus is deaf to 
most of the subtle information content in sentences gener- 
ated by people. 

Such flaws exist in all current natural language systems. For 
the anticipated “naive” user these obstacles must be mini- 
mized or lESIS will prove extremely uncomfortable and 
time-consuming. These flaws could possibly lead to system 
avoidance by less-sophisticated users, thus defeating one of 
the major mission goals. 

Perhaps the simplest way to overcome such problems in 
natural language systems is to restrict the domain of dis- 
course to a small set of possible concepts keyed to known 
individual human differences. Ultimately, the following 
may be the best approach for the “naive” user: Each would 
have a personal identification code known to lESIS, per- 
mitting the system to adjust its language to a compatible 
dialect. Knowledge of customer category could enable 
lESIS to employ reasonable default assumptions in restrict- 
ing the domain of discourse (and thus the vocabulary) to a 
manageable subset of the overall system domain. 

It is important that lESIS be able to communicate at the 
appropriate level of complexity and brevity. To accomplish 
this requires a system capability of modeling individual 
users. Some initial work in this area has been done (Rich, 
1979), but no known current technique offers the level of 
performance necessary for lESIS. Natural language inter- 
facing is one area that requires considerable advancement 
before it can hope to meet the lESIS system requirements: 
domain model, user model, dialogue model, reasonable 
default assumptions and common world knowledge, and 
explanatory capabilities. 

2.2.5 A rtiflcial In telligence Pro blem-Solving 

Clearly lESIS presents the usual difficulties in problem- 
solving typically involved in AI question-answering systems. 
But there is another new and important dimension added - 
effective combination of a world model database, world 


model theory, and a potentially resource-limited observa- 
tional capability. The power of the problem-solvers and 
planners, and their capacity to execute plans in a dynamic 
and only partially known environment, will be instrumental 
in achieving a high-quality information delivery service at 
minimum cost. 

Two specific areas where the quality of problem solving 
affects overall system efficiency and cost were considered 
for illustrative purposes. The first is communication link 
capacity. Given the goal of answering a large number of 
information requests, an intelligent planner able to isolate a 
parsimonious set of observations can considerably reduce 
ground link and intrasatellite link volumes. This set of 
observations is determined by consideration of individual 
requests, the ensemble of all current requests, and a predic- 
tion of expected requests. 

A second area of concern is the number of satellites. If 
the system can employ a very sophisticated theory of obser- 
vation, then it may be possible to shift most data-taking 
tasks to lower resolution. This system would allow data- 
taking by orbiters at higher altitudes having greater fields of 
view; thus a smaller total number of satellites would achieve 
the same frequency of coverage. 

A major lESIS goal is to perform appropriate automatic 
data interpretation. System success in this activity demands 
a high-level capability to understand relationships between 
sensor readings and the actual state of the world as defined 
by human-oriented descriptors. This is precisely the prob- 
lem in visual perception, an active current area in the field 
of artificial intelligence. Section 2.2.6 further discusses 
several aspects of the perception problem for an Earth- 
sensing system, and section 2.2.7 describes the need for 
flexibility and adaptability in lESIS. In both areas ~ per- 
ception, and system flexibility and adaptability - there is a 
tremendous need for development of fully autonomous 
techniques far more powerful than those presently 
available. 

2.2.6 Theory of Observation 

While the number of distinguishable states of the world 
of human interest (at a particular level of resolution and 
description) is extremely large, this figure is still dwarfed by 
the vast number of distinguishable ways the world may 
appear to rudimentary sensors. Machine sensors and human 
eyes see entirely different things when minor changes in the 
world state occur. For instance, in hilly terrain at low sun 
angle, satellites sensor readings vary rapidly as the shadows 
progress, but most of what is of interest to human beings is 
invariant. 

To extract interesting information from sensors it is 
necessary first to understand the nature of the sensor 
as a transducer so that a mathematical inversion process can 
be performed on the readings. This involves computation of 
the electromagnetic reality at the image sensor location 
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(i.e., at the satellite, but before the sensor transduces visual 
photons into a signal). This extracted information is called 
“satellite local reality.” If satellite local reality is not 
obtained, then an interpretation correlation problem arises 
when different sensors are used at various times to observe 
the same thing. 

Once satellite local reality has been determined, it must 
be mapped onto an “object local reality” where “object” 
refers to the surface or volume under observation. For 
example: suppose that the state of the relatively clear atmo- 
sphere and a power spectrum at several chosen frequencies 
for a particular image pixel element of the Earth’s surface 
are accurately known. The theory of observation must be 
able to use the world model (in this case the atmospheric 
component of the model) to determine which parts of the 
satellite local power spectrum are generated from a ground 
effect, and which are atmospheric phenomena. Reflection, 
refraction, absorption and emission cannot, in general, be 
clearly separated. Thus, the attempt to translate satellite 
local to object local descriptions requires some additional 
information. 

The predictive aspects of the world model are important 
here. The model can predict much of the satellite local 
power spectrum, so inversion to object local spectrum need 
not be performed at all except in specific (and hopefully, 
relatively infrequent) cases where observations deviate sig- 
nificantly from prediction. These deviations are called 
anomalies. When an anomaly occurs, there may be simple 
alternative world states lESIS can hypothesize in an 
attempt to find an explanation for the anomaly. In this 
mode of action the model is altered and a new prediction 
for the observation generated. If a reasonable world model 
alteration leads to a predicted image that matches that 
actually observed, then the altered model may represent an 
adequate estimate of current reality. Other specific observa- 
tions should be designed with the objective of testing the 
new hypothesis. 

If an anomaly cannot be disposed of by hypothesizing 
new world states, alternative mechanisms are needed. It is 
desirable to proceed as far as possible without explicit 
human intervention. One alternative is to automatically 
schedule other observational configurations (different 
satellites, sensors, or lighting conditions) to gather enough 
satellite local information and permit clear computation of 
object local signature. Once this is accomplished the final 
interpretation must be made which involves mapping object 
local signatures into a state description suitable for incor- 
poration into the world state component of the world 
model (see section 2.3). 

The role of a predictive model in efficient image gather- 
ing is essential. Even when it is theoretically possible to 
clearly map satellite local reality into a high-level descrip- 
tion of object local reality without such a model, model use 
can significantly increase the efficiency of the observation 
and the speed of the interpretation process. The following 


is a partial list of the many ways world models may be 
helpful: 

• Extending the range of possible viewing conditions 
under which usable information can be gathered (e.g., 
compensating for variations in sun angle) 

• Predicting when certain types of observations are 
impossible because of unfavorable viewing conditions 
which can be known prior to the time of observation 

• Computing the least costly set of sensors (e.g., fewest 
sensors for shortest time, or use of sensors which at 
the particular time have no other demands on them) 
needed to determine a particular fact about the world 

• Avoiding taking certain new observations by deriving 
at least some responses to requests from information 
already in the world model database. 

lESIS is not oriented toward the storage of information 
as images. Rather, images are processed in real time (or 
almost real time) and only extracted information is stored 
in the world model. In such a context, the emphasis shifts 
from finding observation strategies which yield absolutely 
unique sensor signatures for identifying the condition of 
the world, to observing only what is necessary to identify 
the state of the world in the context of the world model. 
The theory of observation can be viewed as part of the 
world model, and represents a large part of the knowledge 
necessary to connect sensor-encoded information with 
more human-oriented descriptions of reality contained in 
the world model database. 

2.2. 7 System Flexibility 

It is very difficult to anticipate the entire range of users 
to whom Earth-sensing systems may be applicable. lESIS 
must be flexible enough to allow a scale-up of total system 
throughput to accommodate a growing number of cus- 
tomers. Similarly, it is unlikely that the mission system will 
have available, by the year 2000, the ultimate in sensor 
technology. Almost certainly, a rapid evolution of ideas and 
technology will occur after a short period of system use. If 
the system is not to be the seed of its own rapid obsoles- 
cence, it is imperative that it be flexible enough to accom- 
modate new modes of observation including new equip- 
ment and new processing procedures. 

The general philosophy of providing a flexible informa- 
tion system for the sophisticated user virtually demands 
that the user be able to specify new algorithms for control- 
ling the data accumulation and data-analysis processes. 
User-defined data collection control becomes important for 
the advanced user when observation scheduling must be 
sensitive to dynamic events in real time - where it would 
be impractical or impossible to use the standard system- 
scheduling mechanisms. Such individuals may require spe- 
cialized data interpretation processing for a variety of 
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reasons including the needs for recognition categories not 
defined as standard categories or for specialized data 
displays. 

The argument can be made that, besides the general 
desirability of system flexibility, it is important to give 
advanced users a flexible, complex, and adaptive tool. Very 
likely some of the most innovative and important lESIS 
applications will arise through the efforts of such 
individuals. 

At present, Landsat data customers represent a relatively 
small but sophisticated population comprised mainly of 
engineers and scientists. lESIS is intended to reach a much 
broader spectrum of potential users, the majority of whom 
are “naive” with regard to computer technology. It is 
imperative that a reasonable model of this target population 
be generated as the system is implemented. Such knowledge 
is necessary for detailed design of user/system interfaces, 
ensuring insertion of user-related elements into the world 
model, and for determining signature analysis and pattern 
recognition techniques required to answer probable user 
questions. 

2.2.8 Data Archiving and Compaction 

The traditional NASA information gathering philosophy 
has been to collect as much raw data as possible from each 
mission and then allow university, industrial, and govern- 
ment researchers to complete the analysis. In the early days 
of space exploration this strategy was reasonable, based on 
spacecraft investment, insofar as it maximized return. But 
today, advancing satellite technology has greatly expanded 
the number of sensors flown and available data rates. The 
resulting torrential flow of information has overwhelmed 
the capacity of the system only 0.05% of data collected 
from space has ever been analyzed. The great unused bulk 
of observations must be stored even though much of it is 
of marginal quality (e.g., obscured by clouds) and probably 
never will be analyzed. 

NASA should consider revising its philosophy of data 
collection to: (1) Make use of knowledge gained from pre- 
vious sensing missions to reduce redundancy. (2) adopt a 
goal-oriented approach to Earth-sensing and other observa- 
tional missions, (3) begin to identify and dispose of poor- 
quality data, (4) condense information as it ages and 
becomes less useful, and (5) present data with full indexing 
and cross-referencing to maximize their utility to the 
consumer. 

Knowledge and experience combined with artificial 
intelligence techniques can eliminate redundancy. For 
example, it is extremely inefficient for an Earth-sensing 
satellite to “rediscover” a lake, road, or city on every 
orbital pass. The truly important aspects of the object are 
its fundamental attributes area, temperature, color, tex- 


ture, etc. - many of which are either constant or predict- 
able. The use of a world model to eliminate continual redis- 
covery of such features could greatly reduce the extraordi- 
nary redundancy of most visual imagery. 

All object attributes studied must reflect worthwhile 
goals. One goal should be the assembly of a historical 
record of Earth features. Others may include more specific 
user-defined objectives. This new emphasis on goal-directed 
observations does not preclude data utilization by the scien- 
tific community in the investigation and verification of new 
theories; quite the contrary, it should actually enhance this 
activity by enabling researchers quickly and easily to direct 
lESIS to collect and process data under closely controlled 
conditions. 

Many time-oriented observations lose some of their value 
with age. After an extended period of time, long-term 
trends are much more useful than individual data values. In 
the proposed lESIS system only the long-term trends are 
retained - original data are eventually discarded. Thus, the 
system processes all data immediately for specific goals and, 
at a later time, integrates trend information into a more 
compact world model representation such as a long-term 
temperature gradient. 

As data are collected in orbit, Earth features and their 
processed image characteristics must be fully indexed to 
sort features and characteristics and to analyze them by 
group. The data then may be manipulated from within a 
fully cross-referenced base. For instance, area type can be 
called out and summed to obtain the total rye acreage in a 
given state. This cross-referencing feature is critical to the 
effectiveness of the Earth-sensing information system. The 
ability to automatically cross-reference and access data by 
content and feature allows rapid aggregation and correla- 
tion, and may promote new research as rapid access to 
useful scientific data becomes routine. The proposed 
database is organized using geographic location as the pri- 
mary key (similar to the World Reference System used for 
Landsat data) with individual features also keyed. Feature 
keys greatly simplify the generation of inverted files 
listing, say, all lakes, deserts, wetlands, forests, or housetop 
areas obvious widespread applications. The detailed 
mechanisms of records layouts, file structures, and database 
languages are beyond the scope of the present study and are 
recommended for future investigation. 

The expense of storing data is a very significant part of 
computer system cost. This consists of direct charges for 
storage media as well as the costs of transferring data to and 
from local and remote storage devices. Data compaction 
and compression produce cost savings by reducing storage 
and transmission requirements. In addition, these data 
reduction methods enable more efficient information 
retrieval and more economical transmission of large quanti- 
ties ot data over computer networks. 
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There are no standard definitions for compaction and 
compression, so the following usages are adopted for the 
purposes of the present study: 

Compaction of data any technique that reduces the 
size of physical data representation while preserving 
the relevant information. 

Compression of data the application of some function 
to elements of the database. 

!f X is a specified element of the database then the com- 
pression of X is V, where >> = /(x). Usually / is invertible, 
which means that the original information may be recov- 
ered whole from the compressed data. 

Compaction techniques other than compression involve 
elimination of information deemed superfluous, in order to 
decrease overall storage requirements. One such method is 
abstraction. Abstraction is accomplished by processing data 
over important common image features (in the case of 
photographic information) by using, for instance, a world 
model. After abstraction it is not possible to recover the 
original image. 

Mission data such as are received daily or are already 
stored in various NASA facilities (e.g., the EROS data 
center at Sioux Falls, South Dakota) and slated for compac- 
tion may be classified within two broad categories con- 
tinuous and noncontinuous depending on timing and 
event characteristics. Classes of continuous data include: 

• Periodic data When the same event appears again 
and again, only one copy need be saved. 

• Trendless data If the data are random continuous, a 
sample should be taken to check for trend. If no 
trend is found, the data may be represented by a 
histogram updated regularly as more inforn^ation 
accumulates. Only histogram parameters need be 
saved. 

• Data with trend If a trend is detected, multiple 
regression and curve-fitting are best to record the 
feature. It may be possible to correlate variation in 
one time series with that in another (e.g., how sea 
level is affected by temperature or pressure). Data 
compression is achieved in this case by fitting poly- 
nomial segments, possibly straight lines, to the data. 

• Data with turning points Turning or ‘‘inflection’ 
points, such as where an upward trend suddenly shifts 
downward in a time series graph, may require differ- 
ent models to be fitted to the two parts of the series. 
Only the parameters of fitted polynomials and a few 
statistical abstractions (e.g., niaxima and minima, 
mean, variance, and several others) in any particular 
range need be saved. 

Suggested classification and processing techniques for non- 
continuous numerical data are summarized in figure 2.7. 


2.3 The World Model 

The world model is a crucial element in the achievement 
of specific goals. Without a sophisticated model two serious 
problems are encountered with remote Earth-sensing data, 
particularly images: 

• It is very difficult, if not impossible in many 
instances, to accurately separate interesting from non- 
interesting observations. 

• It is difficult to comprehend raw sensor data in terms 
readily understandable by human beings. 

The first of these leads to the collection and retention of 
great volumes of data, simply because there is no practical 
way to perform an appropriate selection of the useful sub- 
set of information applicable to a user request. The second 
problem results in gross underutilization even of potentially 
useful data. The lack of a world model in present-day space- 
craft makes necessary a voluminous and costly stream of 
highly redundant data which must be transmitted and col- 
lected on the ground before any useful information is 
retrieved, leaving a huge reservoir of unprocessed data in 
expensive storage facilities. It is the world model which 
transforms lESIS from a collection of remote cameras into 
an entity able to perceive the planet in a manner interesting 
and informative to humans. This world model is a compact 
representation of persistent spatial and temporal character- 
istics of the Earth (its land, oceans, and atmosphere), and 
algorithms for use of the model. 

2. 3. 1 World Model Stnwture 

The lESIS world model has two separate components. 
The first is the state component, which defines the physical 
status of the world to a predetermined level of accuracy 
and completeness at a specified time. Second is the theory 
component that allows derivation of the following informa- 
tion from the state component: (1) Values of parameters of 
the world state not explicitly stored in the state compo- 
nent, and (2) a forecast of the time evolution of the state of 
the world. The theory component gives the world model a 
predictive capability, in that the model can predict facts 
about the world not explicitly retained in the database. 

The disparity between predicted information and reality 
generally increases with increasing computational distance 
separating the starting information and the derived result, 
increasing time in the case of forecasting, and certain other 
factors. The world model requires a continual influx ot new 
observations to remain temporally current. A major 
research goal for efficient lESIS operation is to develop the 
AI capability to construct an effective real-time world 
model which can act as a database for answering questions 
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Figure 2. 7. Classificutiou of and suggested processing techniques for noncontinuous numerical mission data. 




about the state of the world and as a predictive mechanism 
for controlling observation satellites and interpreting 
observations. 

The world model database must contain state compo- 
nent information about the expected character of points on 
the Earth. This includes land use (crop type, urban type, 
etc.) and ground topography. The world model theory 
component must predict some rather ubiquitous changes 
that occur, such as alterations in foliage color and foliage 
loss for certain vegetation areas as a function of season and 
precipitation history; ice formation and melting with the 
seasons; and variations in appearance of rivers from flow 
rate changes due to watershed runoff. As with the ensemble 
of observing satellites, it should be possible initially to set 
the Earth-sensing system into operation with a limited 
world-modeling capability, and later expand this model as 
the technology progresses. 

The key element in handling, processing, and storing 
data in the proposed satellite system is the use of a world 
model to abstract useful information from images and thus 
accomplish a large reduction in required data transmission 
and storage. The model gives the satellite “experience” by 


which to judge new observations. Data compression has 
been investigated in video imagery and, in some cases, 
compressions of 20:1 have been accomplished, but not at 
acceptable fidelity (Graham, 1967). It has been found that 
methods using feature extraction without benefit of a 
world model are capable of compression ratios in excess of 
100:1 (Chien and Peterson, 1977). A world model permits 
still greater data compression by interpreting features and 
their established properties in the full context of the known 
land, sea, and atmospheric environment of the Earth. 

The simplest world model is a flat land map. Figure 2.8 
illustrates such a map of Mildura City in Victoria, Australia, 
representing an area encompassing 1 6 features in 5 distinct 
land types: (1) river, (2) lake, (3) forest, (4) cropland, and 
(5) city area. Each feature is termed a “niche” - a surface 
feature on land or ocean possessing a relatively clear bound- 
ary, common features within that boundary, and whose 
location does not change rapidly with time. On land, niches 
are closely spaced, whereas in the oceans and atmosphere 
they may be quite large. Table 2.4 gives general characteris- 
tics of land, ocean, and atmospheric world models. 


10^ pixels 



VICTORIA, AUSTRALIA 


E141-45 

10® pixels VS. 450 CHARACTERISTICS 
2,222 : 1 REDUCTION 

Figure 2.8. - Sample niche features map of Mildura City, Victoria, Australia. 
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TABLE 2.4.- WORLD MODEL CHARACTERIS- 
TICS OF LAND, OCEAN, AND ATMO- 
SPHERIC WORLD MODELS. 


Land 

Extensive, detailed contour maps exist 

Relatively static 

Sharp boundaries 

Small feature size 

Widely populated 

Oceans 

Gross maps exist 
Constant altitude 
Wide boundaries 
Large feature size 
Sparsely populated 

Atmosphere 
3-dimensional 
Elementary model exists 
Rapidly varying 
Boundaries difficult to identify 
Large shifting patterns 
Some small feature sizes 
Substantially unpopulated 


Niches possess a high degree of spatial redundancy. A 
large lake remains the same lake on each satellite overpass, 
and parts of the lake are very similar to other parts, yet are 
very distinct from the surrounding land. The large redun- 
dancy permits very substantial data reduction by processing 
each sensor reading across a single niche and extracting 
desired characteristics of that niche. These characteristics 
can be combined across sensors to produce yet further 
compaction. Despite the large reduction in physical data, 
no useful information is lost. 

The likely scale of data compaction is illustrated by con- 
sidering a 10 km X 10 km land region at a pixel resolution 
of 10 m. This scale gives an image measuring 1000 X 1000 
pixels. If just one observational wavelength is involved, and 
8 bits are used to represent intensity at each of the 
10^ pixels, the resulting image is 8 X 10^ bits, A limited 
examination of aerial photographs suggests such a region 
will possess an average of 30 niches. Each must be fully 
described in terms of characteristics important to it such as 
area, average sensor value, variance, higher moments, two- 
dimensional sensor intensity gradient, and texture. If 
15 characteristics are sufficient to describe most niches, 
then only 3600 bits (15 characteristics X 30 niches X 
8 bits/niche -characteristic) are needed to replace the 
8,000,000 bits of the full image. A reduction of 2222:1 is 
immediately accomplished. 


Further reduction can be achieved by combining data 
across the approximately 20 sensor wavelengths proposed 
for the satellites, and also across the 15 characteristics. This 
reduced data can be used, for example, in signature identifi- 
cation, specification of niche status, or for classifying an 
anomaly. Near-maximum reduction occurs when imagery is 
required to answer a sophisticated question posing a choice 
from 256 (= 2®) alternatives, an answer requiring just 
8 bits - the 10^ pixel elements over 20 sensors demand one 
8-bit transmission and subsequent storage for a reduction of 
20,000,000:1 ([10^ elements X 20 sensors X 8 bits/sensor- 
element]/ [8 answer bits]). If the question requires a “y^s” 
or “no” answer only 1 bit must be transmitted and the 
absolute maximum data reduction in this simplified exam- 
ple is achieved - 160,000,000:1 - as summarized in 
table 2.5. 

lESIS also is capable of discovery. Novel occurrences can 
be detected by the satellite system when searching for 
anomalous features in the imagery as compared to the 
world model. Many of these anomalies may be boundary 
changes in the existing map, e.g., overflow of the River 
Murray as shown with dark lines in figure 2.9. Others will 
involve unusual values of sensory characteristics of the 
niche (e.g., blight on a corn field). Prompt identification of 
these anomalies allows real-time management action in 
response to the “abnormal” occurrence. Figure 2.10 illus- 
trates a hypothetical set of readouts from an intelligent 
satellite scan over Mildura. 

The most efficient use of a world model requires place- 
ment of a simplified model in memory onboard the satellite 
system (to accomplish direct data reduction) and retention 
of a more sophisticated model in the ground operations 
facility. The latter serves as a master Earth model for use in 
updating, calibrating, and further processing transmitted 
data. (Estimated memory requirements are given in sec- 
tion 2.3.2.) Another very important feature of the ground- 
based world model is that it allows full cross-indexing of 


TABLE 2.5.- DATA REDUCTION USING WORLD 
MODEL IN A SIMPLIFIED EXAMPLE. 


Transmission/storage 

task 

Total 

bits 

Net 

reduction 

10^ image pixels, 20 sensors 
8 bits each 

1.6X10* 

1:1 

30 niches, 20 sensors, 

15 characteristics at 8 bits 

7.2X10^ 

2,222:1 

256-choice answer 

8 

20,000,000:1 

Yes or no answer required 

1 

160,000,000:1 
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10® pixels VS. 450 CHARACTERISTICS 
2,222 : 1 REDUCTION 

Figure 2.9. - Sample niche features map ofMildura Gty; alteration in boundaries of the River Murray due to flooding. 


terrestrial niches, sensor characteristics, and subsidiary 
characteristics. The ground model thus constitutes a full 
working library. A land agent, for instance, easily could 
retrieve moisture content on all corn acreage in southwest 
Iowa from the master files. 

In addition to a physical database memory onboard and 
on the ground, the world model requires extensive artificial 
intelligence software including expert systems which use 
the database in controlling sensing, image location and rec- 
tification, data processing, labeling, anomaly search, and 
decisionmaking on board the satellite and on the ground. A 
set of expert systems is needed to handle overall coordina- 
tion, observation scheduling and user-generated processing 
tasks (fig. 2.1 1). 


2.3.2 Onboard Memory 

Present predictions for onboard memory in the year 
2000 for Earth-sensing satellites are on the order of 
10' bits (Opportunity for Space Exploration to 


Year 2000. Address delivered by A. Adelman at Goddard 
Space Flight Center, 1980). In this section, these projec- 
tions are compared with the storage capacity required for 
lESIS abstracting and onboard processing. The following 
estimates emphasize image processing because this is the 
type of data transmitted at the highest rates - on the order 
of 650 Mb/sec for SAR and 320 Mb/sec for the multilinear 
array (Nagler and Sherry, 1978). 

The bits stored aboard the satellite or satellites for use in 
immediate processing tasks presumably are substantially 
less than the number stored on the ground in lESIS. The 
on-ground world model is a continually updated version of 
the Earth model. Future feasibility experiments plus new 
developments in computer science and technology will 
dictate the specific allocations of memory required in space 
and on the ground. Hence, the following are only crude 
estimates of the storage requirements under a range of 
plausible assumptions. 

For the present work it is assumed that a correlation is 
to be performed between the incoming image and its image 
description stored in the onboard world model. Significant 
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Figure 2.10.- Hypothetical lESIS user printout following scan ofMildura Gty. 


features of the incoming image are contrasted and identi- 
fied with distinguishable features (niches) catalogued in the 
model. The number of bits N needed to store this informa- 
tion on some or all features is A' = ni. where n is the number 
of niches in all or part of the world model and i is the num- 
ber of bits required to describe a niche. Examples that 
estimate N are given below. 

Niches. The niche is a broad distinguishable geographical 
region having some common features across its surface 
when viewed from space. Niches are easily separable, 
somewhat permanent features (large rivers, canals, lakes, 
major highways, cultivated areas or forests) recognizable 


within a predetermined orbital swath. Since niches consist 
of common features, data acquired across them have very 
high redundancy. A large amount of data reduction is 
obtained by describing the entire niche with a limited 
number of common values abstracted from the whole set of 
niche data on record. 

It is estimated that a total of / = 122 bits is required for 
individual niche identification, as follows: Location of 
centroid, 6 bytes, 48 bits; maximum and minimum of 
horizontal, 2 bytes, 16 bits; maximum and minimum verti- 
cal, 2 bytes, 16 bits; orientation, 2 bytes, 16 bits; abstract 
shape among 1000 choices, 10 bits; naming the niche, 
2 bytes, 16 bits. In addition, s sensors are assumed scanning 
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Figure 2.11.- Functions of lESIS expert systems. 


various bands covering a niche having k characteristics 
coded in memory. Thus, 8 sk bits are needed for niche sens- 
ing, and 20 sensors having 15 characteristics require 
2400 bits. It is also conceivable that 10 other characteristics 
might be detailed yielding an additional 80 bits. Conse- 
quently, without using boundary analysis in the image 
correlations, the total information i required for each niche 
is approximately 2600 bits, more than an order of magni- 


tude larger than the information needed simply to identify 
the niche. 

Thus, it is most efficient if the onboard world model 
simply consists of descriptions of those niches the lESIS 
satellite(s) will encounter along an orbital swath on the 
next revolution. The model is uploaded from the ground 
prior to each pass. If swath width in kilometers is W, repeat 
time (in days) for the satellite to cover the entire equatorial 
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Figure 2.12,— Ground-based world model data processing operations. 


region is T, and the number of passes per day (generally 
15-16) is/, then, taking the Earth’s circumference roughly 
as 4X 10^ km, T = (4X If n denotes the number of 

niches in one pass for a given swath width, then 
n = (4X 1 O'* ) IV/niche area . 

For example, if swath width is 330 km and the satellite 
executes 1 5 rev/day , then T is on the order of 8 days. If the 
area of a representative niche is roughly (for simplicity) 
3.3 km^, then n = 4X10^ niches and N- ni = (4X10^) 
(2600)= 1.04X10*® bits. 

Niche at each pixel A gross theoretical estimation is 
obtained by assuming that each pixel element is associated 
with 8 bits of information of t different types. Thus, each 
pixel element plays the role of a niche. If A is the resolu- 
tion of each pixel in km^ and E is the approximate surface 
area of Earth (5.15X10® km^), then the total number of 
bits required to describe pixel elements covering the entire 
planet is ^tE/A. If pixel resolution is 10 m, then 
A = lO*'* km^ /pixel which gives 

(5.15X10® km")/(10-'* kmVpixeI) = 5.15X10*" pixels 


for all of Earth. The total information required to store 
this gross world model is of the order 

8r/:7/l =8f(5. 15X10® )/10"'* =4.12X10*' / bits. 

It appears that a memory capacity onboard a satellite 
of the order 10* * bits is sufficient to allow system opera- 
tion on an orbital swath basis. Presumably between 
10*'*-10*^ bits are needed for ground memory. It is esti- 
mated that by the year 2000 roughly 10* bits of in-space 
memory will be available, so these requirements do not 
appear particularly stringent (Opportunity for Space 
Exploration to Year 2000. Address delivered by A. 
Adelman at Goddard Space Flight Center, 1980). 

2.3.3 Ground-Based World Model Processing 

Processing at the central ground facility can be broken 
into paths as shown in figure 2.12. Action items which the 
system should generate in response to observation of an 
anomaly are carried out, appropriate alarms sounded and 
individuals notified. Anomalies that the system cannot 
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identify undergo close human scrutiny — if subsequently 
identified, the expected anomaly/action file is updated; if 
not, the event is catalogued and correlated with other 
similar but also unidentified anomalies. Identified anoma- 
lies keyed to time of occurrence, intensity, and other 
important parameters are archived and used to update the 
world model if necessary. Data abstracted from observed 
features are archived and used for world model updating. 
Information gathered in response to user requests is trans- 
mitted to customers for their exclusive use and is not 
automatically stored by the system. 

2.4 Autonomous Satellites 

This section focuses on individual satellites and the 
collective lESIS system, as suggested in figure 2.13. To 
satisfy overall lESIS goals, all satellites must be equipped 
with an appropriate ensemble of sensors and reside in orbits 
providing sufficiently frequent observation opportunities 


for all points of interest. Just as important, however, is the 
ability of each device to accept brief high-level instructions 
to guide its observations, and to perform massive onboard 
processing for abstraction of high-level information. 

2 A. I Onboard Procemng 

The observing satellite receives the schedule for its next 
pass from the uplink (section 2.4.2) and performs the 
requisite processing according to this instruction set. 
Terrestrial cloud cover and weather conditions are obtained 
from one of the lESIS geostationary satellites and naviga- 
tional data are transmitted from a global positioning system 
already in place. Each satellite adjusts its attitude as 
required, turns detectors on and off, modifies sensor 
resolution, and takes both active and passive observations 
of the Earth. Data then are processed by comparison to 
predicted observables as derived from the world model. 
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Anomalies are identified if possible, catalogued, and appro- 
priate action taken. Anomalies that cannot reliably be 
identified by lESIS are placed on file for transmission to a 
human analyst for interpretation and action. All sensor data 
are abstracted and summarized by feature and placed in the 
abstracted observation file for archiving and world model 
updating. Any user-requested processing is then performed 
and a user file established. These processes are shown 
schematically in figure 2.14. 


2.4.2 Uplink and Downlink 

The uplink sends detailed observational and processing 
schedules to lESIS satellites. A possible component of the 
uplink package is a set of sensor values expected to be 
observed during the next set of observations. This expecta- 
tion is generated by using the world model as the standard 
against which to compare sensor observations so that 
anomalies may be identified. The uplink is illustrated in 
figure 2.15. Information needed during the next observa- 
tion period is transmitted from the central processing 
facility via ground station and geosynchronous communica- 
tions satellite up to the observation satellite as discussed in 
section 2.2. 

The downlink reverses the uplink process by consolidat- 
ing data gathered by satellite and returning them to the 
ground in one transmission. The downlink also uses the geo- 
synchronous communications link to transmit data to a 


ground station which relays the information to the central 
processing facility. The process of downlink consolidation 
is illustrated in figure 2,16. 

2.4.3 Image Processing 

A primary lESIS requirement is the necessity to perform 
rapid and massive data reduction aboard the satellite in the 
sequence suggested in table 2.6. The focus is on image data 
acquired at high rates - presently 120 Mb/sec for SIR, 
85 Mb/sec for the Thematic Mapper, and in excess of 
600 Mb/sec for SAR systems (Nagler and Sherry, 1978). 
Such rates may arise in each of perhaps 20 sensors in 
certain extreme cases of lESIS operations, which requires 
that the spacecraft carry onboard high-speed processors. 

lESlS high-speed processors might evolve from faster 
serial logic devices, e.g., those which may be developed 
from Josephson tunnel technology. However, an alternative 
approach is the use of parallel logic to perform many opera- 
tions simultaneously. By executing thousands of computa- 
tions at once an intrinsic speed advantage equal to the 
number of individual processors is theoretically possible. In 
practice, this hypothetical limit may not be attainable due 
to pragmatic technological restrictions on each individual 
element in an array of thousands of processors. In spite of 
this, computing speeds within two orders of magnitude of 
the theoretical limit have already been obtained (Schaefer, 
1980). Data handling using thousands of active elements 
simultaneously is called “parallel processing.” 
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Preprocessing functions including extremely rapid recti- 
fication of images, formatting, noise removal, imbalance, 
and radiometric correcting can be performed best by 
manipulating data in parallel rather than serially. Projec- 
tions for serial onboard image processing range up to 
10^ operations per second (Opportunity for Space Explora- 
tion to Year 2000. Address delivered by A. Adelman at 
Goddard Space Flight Center, 1980), whereas, those for 
parallel processors extend up to 10* * operations per second 


per pixel (assuming 100 clock cycles per operation). 
Another capability expected to be using parallel processing 
is a 1000 X 1000 parallel input array operating at up to 
100 MHz clock rates (James Strong, personal communica- 
tion, 1980). In one proposed 1000 X 1000 input array, as 
shown in figure 2.17, threshold photosensors on a wafer 
lead directly to massively parallel processing elements 
which in turn are connected (also through the wafers) to 
memories. Processing rates expected for massively parallel 
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Figure 2.14,- Onboard processing flowchart. 
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TABLE 2.6.- ONBOARD DATA REDUCTION 
PROCESSES. 


processors (MPP) are tabulated in table 2.7. Further infor- 
mation and technical descriptions are available in Gilmore 
et al.(1979). 


Preprocessing 

Name image region 
Format image data 
Apply sensor corrections 

Processing 

Cross-correlate image and map 
Geometric correction 
Resample data 
Obtain niche boundary 
Generate niche mask 
Process niche data 
Combine data across sensors 
Detect and characterize anomalies 

Postprocessing 

Attach niche labels 
User tags 
Assign priorities 
Update data base 


TABLE 2.7.- ESTIMATED SPEED OF COMPU- 
TATIONS USING MASSIVELY PARALLEL 
PROCESSOR. 


Function 

10® Hz clock 

Area 

6 Msec 

Average 

50 Msec 

Variance 

0.1 1 msec 

Slope 

1 .4 Msec 

Fourier transform 

1 1 .7 msec 

Histogram 

1 .5 msec 

Classification 

0.6 msec 

Matching two images 

8 1 .8 msec 

Resampling 

0.05 msec 



Figure 2.15.— lESIS uplink processing. 







2AA Sensor Requirements 

lESIS sensor requirements are dictated by the specific 
terrestrial environment that must be scanned to fulfill an 
Earth resource mission. Full utilization of satellite capabili- 
ties and operating time demands useful sensing during 
daylight and nighttime passes of an orbit and during cloud 
cover. 

Daylight operation involves observation by sunlight fil- 
tered through atmosphere twice before detection by satel- 
lite sensors. Most filtered sunlight lies in the visible region 
extending somewhat into the near-UV and further into the 
near-infrared. An optimum match between the daylight 
Earth environment and the satellite passive scanning system 
must include a visible, near-infrared, and some ultraviolet 
detection capability. A properly chosen 3-dimensional color 
space obtained using a red, green, and blue filter set yields 
color discrimination roughly comparable to that of the 
human eye. Detection at the chlorophyll absorption region 
near 650 nm gives useful discrimination for vegetation, 
while for water detection the low reflectivity region near 
850 nm is useful (Schappell and Tietz, 1979). A pair of UV, 
five visible, and three near-IR bands should provide suffi- 
ciently broad color space (10 dimensions) to allow very 
widespread signature analysis of important terrestrial fea- 
tures such as crops, rivers, lakes, clouds, forests, and snow 
covers. 


The nighttime environment may be scanned passively for 
thermal radiation at a temperature near 300 K. The black- 
body emissions of the cool Earth peak in the far-lR near 
10 jum. Four wideband far-IR sensors would allow accurate 
temperature and signature definition of nighttime features, 
although not to the same precision and resolution as with 
daytime sensing. 

All-weather capability requires active microwave scan- 
ning of the Earth. The Synthetic Aperture Radar (SAR) 
operating at 1-10 GHz (Nagler and Sherry, 1978; OAST, 
1980) is capable of essentially all-weather observation at 
good resolution. The SAR system also would augment 
nighttime passive measurement in the far-IR. 

Altitude sensing provides useful information about ter- 
restrial resources such as crop height, reservoir levels, or 
mountain snow cover. Height is recorded from differential 
altitude measurements performed at a boundary, e.g., by 
comparing the heights of crop tops to nearby level ground. 
A differential altitude measurement system is possible using 
rapid Q-switched LIDAR. Absolute altitude measurements 
can be obtained by LIDAR or microwave altimeter. Differ- 
ential velocity measurements at a boundary (e.g., a river 
bank) can be taken by Doppler shift analysis. 

Undoubtedly there will be requirements for additional 
specialized optical, infrared, and microwave sensor bands to 
detect important surface and atmospheric components such 
as ozone, water, water vapor, and carbon dioxide (Golovsko 
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orbital speed is 8(2)(10X7 km/sec)(l 10 km)/(5X 10 ^ km)^ 
+ 8(1X20)(7 km/secX330 km)/(15X10-^ km)^ = 6.6X10^ 
bits/sec. Of the remaining sensors the SAR will produce the 
maximum data rate by far. Today’s SAR apparatus gener- 
ates data at 0.65X10’ bits/sec (Nagler and Sherry, 1978), 
Representative Doppler LIDAR, Doppler radar, and laser 
altimeters return data at the rate of several tens of kilobits 
per second. Thus, onboard computing capability require- 
ments must be sufficient to handle data rates near 
7X10’ bits/sec. This is roughly an order of magnitude 
higher than that used in present Landsat orbiters. 

2.4,6 Satellite Requirements 

A summary of required measurement rates has been 
provided by Nagler and Sherry (1978) for a wide range of 
environmental and resource assessments. The necessary 
frequency of observation generaUy is lowest for land-based 
features, higher for ocean observation, and highest for 
atmospheric and weather assessments, with considerable 
overlap in the requirements. Table 2.9 indicates the fre- 
quencies of Earth observation and attendant swath widths 
believed reasonable for the lESIS system. 

IMAGE 



and Pakhomov, 1978). Somewhat artibrarily, six specialized 
sensor bands have been allocated to these uses for purposes 
of the present study. 

Sensors are configured into a wide-angle medium- 
resolution sensor array and additionally into at least two 
arrays of narrow-angle, high -re solution sensors capable of 
independent accurate aiming over at least the swath width 
of the wider-angle array. Narrow arrays allow independent 
coverage of terrestrial features needed to satisfy conflicting 
requests during a particular orbital pass. They also may be 
used to obtain stereoscopic imagery, say, of cloud tops, by 
setting one array to view forward and the other aft during a 
pass and later combining the two image streams appro- 
priately. Table 2.8 summarizes one possible sensor set. 

2A.5 Data Rate Estimation 

The bit rate generated by one optical or IR sensor is 
given approximately by %SBVWjA, where S is the number 
of sensor sets, B is bands per sensor set, V is orbital veloc- 
ity, W is swath width, and A is pixel resolution in km^ for 
an 8-bit pixel. For two sets of narrow sensors and one set of 
wide sensors the total bit rate generated at a 7 km/sec 


Figure 2,17, — A projected advanced massively parallel processing system. 
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TABLE 2.8.- SENSOR CONFIGURATION OF 
AN EARTH OBSERVING SATELLITE. 


Configuration and array of a possible set of active and 
passive sensors in an early mission. All sensor bands 
to be operated simultaneously if desirable. 

Configuration 

1 set of wide angle sensors comprising the full 
array of sensors to scan 330 km swath at 15 m reso- 
lution, or as limited by individual sensor. 

2 sets of narrow angle UV, visible and near IR 
sensors, capable of accurate aiming, to cover 1 10 km 
swath at 5 m resolution; '^7X 10^ bits/sec. 

Sensor Array 

10 bands - UV, visible and near IR (daylight) 

4 bands — far IR (night) 

6 specialized bands (atmospheric composition) 
SAR (all weather) 

LIDAR 

Differential height 
Differential velocity 
Altimeter 


The lESIS satellite program is envisioned as developing 
in a long-term sequence carrying well into the next century. 
A detailed world model of land features already exists as 
contour maps covering a significant portion of the conti- 
nents. Land features bave sharp boundaries and vary only 
slowly over time. Oceans have wider geographic features 
that vary seasonally. The atmosphere requires three- 
dimensional modeling of rapidly varying phenomena. An 
obvious difference between land and ocean or atmosphere 
from a user standpoint is the large human population on 
land and its virtual absence elsewhere. Table 2.4 summar- 
izes the characteristics of world models of the land, oceans, 
and atmosphere. 

The logical deployment sequence of user-oriented 
resource satellites begins with a set of basic land-observing 
satellites whose world model already can be rather fully 
detailed. Since the satellites will spend about 75% of their 
time over the ocean it is natural to include ocean-sensing 
capability with as much ocean modeling as is feasible at the 
time of design and launch. Atmospheric sensing and rudi- 
mentary modeling should be included, both for understand- 
ing the state of the atmosphere and also as a necessary part 
of the interpretation process for sensor readings of land and 
ocean observations. 

To assure long life for these sophisticated satellites, 
reasonably high orbits are required. Atmospheric path dis- 


TABLE 2.9.- POSTULATED OBSERVATION 
FREQUENCY AND SWATH WIDTHS. 


Niche features 
observed 

Observation 
frequency, 
per day 

Maximum 
swath width, 
km 

Land 

0.5 

350 

Ocean 

3 

700 

Atmosphere 

12 

1 

1400 


tortion and sun angle introduce errors and complications 
into the interpretation process for imaging data. Path 
distortion causes reddening and other wavelength- 
dependent absorptions, and Rayleigh and Mie scattering are 
especially sensitive to particle size in the atmosphere and to 
sun angles. 

The use of sun-synchronous satellites simplifies the situa- 
tion considerably, a rational initial constraint which could 
be removed at some later time when more sophisticated 
modeling becomes available. It appears reasonable to have 
a set of sun-synchronous satellites operating continuously 
so that each Earth ground point is covered at the equator 
every 2 days by at least one satellite of the set. Sun- 
synchrony produces roughly the same sun-angle condi- 
tions over an observed land point for a particular satellite 
and helps to standardize image interpretation for that 
satellite at that point. An orbit near present-day Landsat 
altitudes (920 km, nominal) will support a long-lived 
satellite. If altitude is adjusted to a 14-1/8 rev/day rate, the 
ground trace of a particular satellite repeats every 8 days. 
Four such satellites could cover the Earth with the desired 
2-day period. 

The swath width required of a satellite for 8-day cover- 
age at 14-1/8 rev/day is about 350 km. However, in order 
to take account of partial cloud cover the team proposes 
six satellites in sun-synchronous orbits. If these are placed 
substantially uniformly about the Earth’s circumference 
the local viewing times for each satellite are spaced about 
2 hr apart. Bunching may be desirable if there are reasons 
to pick a particular local viewing time. Polar conditions can 
be monitored by a seventh polar satellite, which may also 
act as a spare if one of the sun -synchronous satellites is 
disabled. 

To relay data to the continental United States, two geo- 
stationary satellites are required. These satellites are also 
used to monitor global conditions, particularly cloud cover. 
Global cover information is compiled by lESIS to prepare 
each satellite for the tasks it can most usefully perform 
during its upcoming orbit, by enabling modifications in 
sensors and processing to optimize the information 
obtained from each series of observations. 

Ocean coverage of a particular ocean point three times 
per day with a 700-km swath width requires 12 satellites, 
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and atmospheric coverage at a rate of 12 times per day with 
a 1400 km swath requires 24 satellites, each with an 8-day 
repeat cycle (assuming the same sun-synchronous orbital 
parameters given above for land-observing satellites). 

The technology available in the year 2000 (hypothetical 
lESIS deployment date) will, of course, dictate the actual 
satellite configuration employed. Still, the initial set of 
satellites should emphasize land observation with more 
sophisticated oceanic and atmospheric satellites phased in 
as the ability to model these systems develops. Figure 2.18 
summarizes the basically land-observing system described 
earlier. 

2.5 Time Phasing 

The intelligent Earth-sensing information system pro- 
posed herein is an evolutionary system which considerably 
extends both planned and existing NASA missions. The 



Figure 2.18. - Orbit characteristics: Possible initial lESIS 
satellite configuration. 


time-phasing chart in figure 2.19 is oriented to development 
of the various components culminating in an operational 
system by the year 2000. Little attention was given to 
sensor development as this technology is driven by the 
various demands of other users and by general progress in 
this technical area (Breckenridge and Husson, 1979). Most 
attention was directed to software and artificial intelligence 
development as these lie at the heart of lESIS, although 
advanced hardware technology R&D also is required to 
achieve high packing densities, large wafers, fault-tolerant 
designs, advanced cooling techniques, advanced intercon- 
nections, more logic functions between array elements, 
advanced data output, and parallel input from sensors to 
buffer memory. Some of the major points are as follows; 

• Research into automatic mapping and world model 
development should begin early. A world model for 
use onboard should be ground-demonstrated by mid- 
1987 and a Shuttle demonstration of the world 
model/sensor operation completed by 1990 to meet 
the projected lESlS deployment date (2000 AD). 

• Parallel processor development should be given high 
priority. The Massively Parallel Processor (a 
128 X 128 array processor) will be operational in 
1982. A 1000 X 1000 (or perhaps a 10,000 X 100) 
array for parallel processing should be developed by 
1990 and should be flown on a Shuttle test satellite 
by 1995 to meet the 2000 AD deadline. 

• Natural language user interfacing with the data sys- 
tem should be operational by 1990. 

• Development of a model of the user population 
should begin immediately and be phased with natural 
language and world model development. The prospec- 
tive ground demonstration of the world model using 
direct data from an advanced Landsat can be made 
available at some point to selected users on an experi- 
mental basis. The information on prospective selected 
users can form a preliminary user model. 

• Signature analysis, data handling, and security will 
require continuing development and algorithm refine- 
ment. By 1995, software should be flown onboard 
both experimentally and as an initial phase-in on the 
autonomous satellites. 

• A large world model encompassing terrestrial, oceano- 
graphic, and atmospheric components and a satellite 
system scheduler/controller should be ready by the 
year 2000. 

• A gradual phase-out of Landsat D orbiters and 
phase-in of more autonomous “smart"’ satellites 
should begin. By the year 2000, fully autonomous 
satellites carrying world models should be available 
for long-term operation and initiation of the com- 
plete lESIS program. 
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Figure 2.19.- lESlS time phasing chart, 1980-2020. 
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